{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Example Seldon Core Deployments using Helm\n",
    "<img src=\"images/deploy-graph.png\" alt=\"predictor with canary\" title=\"ml graph\"/>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Prerequistes\n",
    "You will need\n",
    " - [Git clone of Seldon Core](https://github.com/SeldonIO/seldon-core)\n",
    " - A running Kubernetes cluster with kubectl authenticated\n",
    " - [python grpc tools](https://grpc.io/docs/quickstart/python.html)\n",
    " - [Helm client](https://helm.sh/)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Creating a Kubernetes Cluster\n",
    "\n",
    "Follow the [Kubernetes documentation to create a cluster](https://kubernetes.io/docs/setup/).\n",
    "\n",
    "Once created ensure ```kubectl``` is authenticated against the running cluster."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Setup"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "namespace/seldon created\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl create namespace seldon"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Context \"minikube\" modified.\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl config set-context $(kubectl config current-context) --namespace=seldon"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "clusterrolebinding.rbac.authorization.k8s.io/kube-system-cluster-admin created\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl create clusterrolebinding kube-system-cluster-admin --clusterrole=cluster-admin --serviceaccount=kube-system:default"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Install Helm"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "serviceaccount/tiller created\n",
      "clusterrolebinding.rbac.authorization.k8s.io/tiller created\n",
      "$HELM_HOME has been configured at /home/clive/.helm.\n",
      "\n",
      "Tiller (the Helm server-side component) has been installed into your Kubernetes Cluster.\n",
      "\n",
      "Please note: by default, Tiller is deployed with an insecure 'allow unauthenticated users' policy.\n",
      "To prevent this, run `helm init` with the --tiller-tls-verify flag.\n",
      "For more information on securing your installation see: https://docs.helm.sh/using_helm/#securing-your-helm-installation\n",
      "Happy Helming!\n"
     ]
    }
   ],
   "source": [
    "!kubectl -n kube-system create sa tiller\n",
    "!kubectl create clusterrolebinding tiller --clusterrole cluster-admin --serviceaccount=kube-system:tiller\n",
    "!helm init --service-account tiller"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Waiting for deployment \"tiller-deploy\" rollout to finish: 0 of 1 updated replicas are available...\n",
      "deployment \"tiller-deploy\" successfully rolled out\n"
     ]
    }
   ],
   "source": [
    "!kubectl rollout status deploy/tiller-deploy -n kube-system"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Start seldon-core"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "NAME:   seldon-core-crd\n",
      "LAST DEPLOYED: Sun Oct 14 20:49:40 2018\n",
      "NAMESPACE: seldon\n",
      "STATUS: DEPLOYED\n",
      "\n",
      "RESOURCES:\n",
      "==> v1beta1/ClusterRoleBinding\n",
      "NAME                        AGE\n",
      "seldon-spartakus-volunteer  0s\n",
      "\n",
      "==> v1/ConfigMap\n",
      "NAME                     DATA  AGE\n",
      "seldon-spartakus-config  3     0s\n",
      "\n",
      "==> v1beta1/CustomResourceDefinition\n",
      "NAME                                         AGE\n",
      "seldondeployments.machinelearning.seldon.io  0s\n",
      "\n",
      "==> v1beta1/Deployment\n",
      "NAME                        DESIRED  CURRENT  UP-TO-DATE  AVAILABLE  AGE\n",
      "seldon-spartakus-volunteer  1        0        0           0          0s\n",
      "\n",
      "==> v1/ServiceAccount\n",
      "NAME                        SECRETS  AGE\n",
      "seldon-spartakus-volunteer  1        0s\n",
      "\n",
      "==> v1beta1/ClusterRole\n",
      "NAME                        AGE\n",
      "seldon-spartakus-volunteer  0s\n",
      "\n",
      "\n",
      "NOTES:\n",
      "NOTES: TODO\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "!helm install ../helm-charts/seldon-core-crd --name seldon-core-crd --set usage_metrics.enabled=true"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "NAME:   seldon-core\n",
      "LAST DEPLOYED: Sun Oct 14 20:49:41 2018\n",
      "NAMESPACE: seldon\n",
      "STATUS: DEPLOYED\n",
      "\n",
      "RESOURCES:\n",
      "==> v1/ClusterRoleBinding\n",
      "NAME    AGE\n",
      "seldon  1s\n",
      "\n",
      "==> v1/ServiceAccount\n",
      "NAME    SECRETS  AGE\n",
      "seldon  1        1s\n",
      "\n",
      "==> v1beta1/ClusterRole\n",
      "NAME        AGE\n",
      "seldon-crd  1s\n",
      "\n",
      "==> v1beta1/Role\n",
      "ambassador    1s\n",
      "seldon-local  1s\n",
      "\n",
      "==> v1beta1/RoleBinding\n",
      "NAME        AGE\n",
      "ambassador  1s\n",
      "\n",
      "==> v1/RoleBinding\n",
      "seldon  1s\n",
      "\n",
      "==> v1/Service\n",
      "NAME                          TYPE       CLUSTER-IP      EXTERNAL-IP  PORT(S)                        AGE\n",
      "seldon-core-ambassador-admin  NodePort   10.111.210.204  <none>       8877:31652/TCP                 1s\n",
      "seldon-core-ambassador        NodePort   10.101.200.250  <none>       8080:32625/TCP                 1s\n",
      "seldon-core-seldon-apiserver  NodePort   10.110.40.180   <none>       8080:30975/TCP,5000:32443/TCP  1s\n",
      "seldon-core-redis             ClusterIP  10.105.48.94    <none>       6379/TCP                       1s\n",
      "\n",
      "==> v1beta1/Deployment\n",
      "NAME                                DESIRED  CURRENT  UP-TO-DATE  AVAILABLE  AGE\n",
      "seldon-core-ambassador              1        1        1           0          1s\n",
      "seldon-core-seldon-apiserver        1        1        1           0          1s\n",
      "seldon-core-seldon-cluster-manager  1        1        1           0          1s\n",
      "seldon-core-redis                   1        1        1           0          1s\n",
      "\n",
      "==> v1/Pod(related)\n",
      "NAME                                                 READY  STATUS             RESTARTS  AGE\n",
      "seldon-core-ambassador-778c58bf5d-k2h66              0/2    ContainerCreating  0         1s\n",
      "seldon-core-seldon-apiserver-6b8dbc978b-gh5nh        0/1    ContainerCreating  0         1s\n",
      "seldon-core-seldon-cluster-manager-596d4674fd-fhj7h  0/1    ContainerCreating  0         1s\n",
      "seldon-core-redis-8668565565-t2824                   0/1    ContainerCreating  0         1s\n",
      "\n",
      "\n",
      "NOTES:\n",
      "Thank you for installing Seldon Core.\n",
      "\n",
      "Documentation can be found at https://github.com/SeldonIO/seldon-core\n",
      "\n",
      "\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "!helm install ../helm-charts/seldon-core --name seldon-core --namespace seldon  --set ambassador.enabled=true"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Waiting for deployment \"seldon-core-seldon-cluster-manager\" rollout to finish: 0 of 1 updated replicas are available...\n",
      "deployment \"seldon-core-seldon-cluster-manager\" successfully rolled out\n",
      "deployment \"seldon-core-seldon-apiserver\" successfully rolled out\n",
      "Waiting for deployment \"seldon-core-ambassador\" rollout to finish: 0 of 1 updated replicas are available...\n",
      "deployment \"seldon-core-ambassador\" successfully rolled out\n"
     ]
    }
   ],
   "source": [
    "!kubectl rollout status deploy/seldon-core-seldon-cluster-manager\n",
    "!kubectl rollout status deploy/seldon-core-seldon-apiserver\n",
    "!kubectl rollout status deploy/seldon-core-ambassador"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Set up REST and gRPC methods\n",
    "\n",
    "**Ensure you port forward ambassador**:\n",
    "\n",
    "```\n",
    "kubectl port-forward $(kubectl get pods -n seldon -l service=ambassador -o jsonpath='{.items[0].metadata.name}') -n seldon 8003:8080\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Install gRPC modules for the prediction protos."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "!cp ../proto/prediction.proto ./proto\n",
    "!python -m grpc.tools.protoc -I. --python_out=. --grpc_python_out=. ./proto/prediction.proto"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Serve Single Model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "NAME:   mymodel\n",
      "LAST DEPLOYED: Sun Oct 14 20:51:00 2018\n",
      "NAMESPACE: seldon\n",
      "STATUS: DEPLOYED\n",
      "\n",
      "RESOURCES:\n",
      "==> v1alpha2/SeldonDeployment\n",
      "NAME     AGE\n",
      "mymodel  0s\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "!helm install ../helm-charts/seldon-single-model --name mymodel"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[04m\u001b[31;01m-\u001b[39;49;00m\u001b[04m\u001b[31;01m-\u001b[39;49;00m\u001b[04m\u001b[31;01m-\u001b[39;49;00m\r\n",
      "\u001b[04m\u001b[31;01m#\u001b[39;49;00m \u001b[04m\u001b[31;01mS\u001b[39;49;00m\u001b[04m\u001b[31;01mo\u001b[39;49;00m\u001b[04m\u001b[31;01mu\u001b[39;49;00m\u001b[04m\u001b[31;01mr\u001b[39;49;00m\u001b[04m\u001b[31;01mc\u001b[39;49;00m\u001b[04m\u001b[31;01me\u001b[39;49;00m\u001b[04m\u001b[31;01m:\u001b[39;49;00m \u001b[04m\u001b[31;01ms\u001b[39;49;00m\u001b[04m\u001b[31;01me\u001b[39;49;00m\u001b[04m\u001b[31;01ml\u001b[39;49;00m\u001b[04m\u001b[31;01md\u001b[39;49;00m\u001b[04m\u001b[31;01mo\u001b[39;49;00m\u001b[04m\u001b[31;01mn\u001b[39;49;00m\u001b[04m\u001b[31;01m-\u001b[39;49;00m\u001b[04m\u001b[31;01ms\u001b[39;49;00m\u001b[04m\u001b[31;01mi\u001b[39;49;00m\u001b[04m\u001b[31;01mn\u001b[39;49;00m\u001b[04m\u001b[31;01mg\u001b[39;49;00m\u001b[04m\u001b[31;01ml\u001b[39;49;00m\u001b[04m\u001b[31;01me\u001b[39;49;00m\u001b[04m\u001b[31;01m-\u001b[39;49;00m\u001b[04m\u001b[31;01mm\u001b[39;49;00m\u001b[04m\u001b[31;01mo\u001b[39;49;00m\u001b[04m\u001b[31;01md\u001b[39;49;00m\u001b[04m\u001b[31;01me\u001b[39;49;00m\u001b[04m\u001b[31;01ml\u001b[39;49;00m\u001b[04m\u001b[31;01m/\u001b[39;49;00m\u001b[04m\u001b[31;01mt\u001b[39;49;00m\u001b[04m\u001b[31;01me\u001b[39;49;00m\u001b[04m\u001b[31;01mm\u001b[39;49;00m\u001b[04m\u001b[31;01mp\u001b[39;49;00m\u001b[04m\u001b[31;01ml\u001b[39;49;00m\u001b[04m\u001b[31;01ma\u001b[39;49;00m\u001b[04m\u001b[31;01mt\u001b[39;49;00m\u001b[04m\u001b[31;01me\u001b[39;49;00m\u001b[04m\u001b[31;01ms\u001b[39;49;00m\u001b[04m\u001b[31;01m/\u001b[39;49;00m\u001b[04m\u001b[31;01mm\u001b[39;49;00m\u001b[04m\u001b[31;01mo\u001b[39;49;00m\u001b[04m\u001b[31;01md\u001b[39;49;00m\u001b[04m\u001b[31;01me\u001b[39;49;00m\u001b[04m\u001b[31;01ml\u001b[39;49;00m\u001b[04m\u001b[31;01m.\u001b[39;49;00m\u001b[04m\u001b[31;01mj\u001b[39;49;00m\u001b[04m\u001b[31;01ms\u001b[39;49;00m\u001b[04m\u001b[31;01mo\u001b[39;49;00m\u001b[04m\u001b[31;01mn\u001b[39;49;00m\r\n",
      "{\r\n",
      "    \u001b[34;01m\"apiVersion\"\u001b[39;49;00m: \u001b[33m\"machinelearning.seldon.io/v1alpha2\"\u001b[39;49;00m,\r\n",
      "    \u001b[34;01m\"kind\"\u001b[39;49;00m: \u001b[33m\"SeldonDeployment\"\u001b[39;49;00m,\r\n",
      "    \u001b[34;01m\"metadata\"\u001b[39;49;00m: {\r\n",
      "        \u001b[34;01m\"labels\"\u001b[39;49;00m: {\r\n",
      "            \u001b[34;01m\"app\"\u001b[39;49;00m: \u001b[33m\"seldon\"\u001b[39;49;00m\r\n",
      "        },\r\n",
      "        \u001b[34;01m\"name\"\u001b[39;49;00m: \u001b[33m\"mymodel\"\u001b[39;49;00m\r\n",
      "    },\r\n",
      "    \u001b[34;01m\"spec\"\u001b[39;49;00m: {\r\n",
      "        \u001b[34;01m\"name\"\u001b[39;49;00m: \u001b[33m\"mymodel\"\u001b[39;49;00m,\r\n",
      "        \u001b[34;01m\"predictors\"\u001b[39;49;00m: [\r\n",
      "            {\r\n",
      "                \u001b[34;01m\"componentSpecs\"\u001b[39;49;00m: [{\r\n",
      "                    \u001b[34;01m\"spec\"\u001b[39;49;00m: {\r\n",
      "                        \u001b[34;01m\"containers\"\u001b[39;49;00m: [\r\n",
      "                            {\r\n",
      "                                \u001b[34;01m\"image\"\u001b[39;49;00m: \u001b[33m\"seldonio/mock_classifier:1.0\"\u001b[39;49;00m,\r\n",
      "                                \u001b[34;01m\"imagePullPolicy\"\u001b[39;49;00m: \u001b[33m\"IfNotPresent\"\u001b[39;49;00m,\r\n",
      "                                \u001b[34;01m\"name\"\u001b[39;49;00m: \u001b[33m\"classifier\"\u001b[39;49;00m,\r\n",
      "                                \u001b[34;01m\"resources\"\u001b[39;49;00m: {\r\n",
      "                                    \u001b[34;01m\"requests\"\u001b[39;49;00m: {\r\n",
      "                                        \u001b[34;01m\"memory\"\u001b[39;49;00m: \u001b[33m\"1Mi\"\u001b[39;49;00m\r\n",
      "                                    }\r\n",
      "                                }\r\n",
      "                            }\r\n",
      "                        ],\r\n",
      "                        \u001b[34;01m\"terminationGracePeriodSeconds\"\u001b[39;49;00m: \u001b[34m1\u001b[39;49;00m\r\n",
      "                    }}\t\t\t\t   \r\n",
      "\t        ],\r\n",
      "                \u001b[34;01m\"graph\"\u001b[39;49;00m:\t\t\t \r\n",
      "\t\t{\r\n",
      "                    \u001b[34;01m\"children\"\u001b[39;49;00m: [],\r\n",
      "                    \u001b[34;01m\"name\"\u001b[39;49;00m: \u001b[33m\"classifier\"\u001b[39;49;00m,\r\n",
      "                    \u001b[34;01m\"endpoint\"\u001b[39;49;00m: {\r\n",
      "\t\t\t\u001b[34;01m\"type\"\u001b[39;49;00m : \u001b[33m\"REST\"\u001b[39;49;00m\r\n",
      "\t\t    },\r\n",
      "                    \u001b[34;01m\"type\"\u001b[39;49;00m: \u001b[33m\"MODEL\"\u001b[39;49;00m\r\n",
      "                },\t\t\t \t\t\t \r\n",
      "                \u001b[34;01m\"name\"\u001b[39;49;00m: \u001b[33m\"mymodel\"\u001b[39;49;00m,\r\n",
      "                \u001b[34;01m\"replicas\"\u001b[39;49;00m: \u001b[34m1\u001b[39;49;00m,\r\n",
      "\t\t\u001b[34;01m\"labels\"\u001b[39;49;00m: {\r\n",
      "\t\t    \u001b[34;01m\"version\"\u001b[39;49;00m : \u001b[33m\"v1\"\u001b[39;49;00m\r\n",
      "\t\t}\r\n",
      "            }\r\n",
      "        ]\r\n",
      "    }\r\n",
      "}\r\n"
     ]
    }
   ],
   "source": [
    "!helm template ../helm-charts/seldon-single-model --name mymodel | pygmentize -l json"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Waiting for deployment \"mymodel-mymodel-classifier-0\" rollout to finish: 0 of 1 updated replicas are available...\n",
      "Waiting for deployment spec update to be observed...\n",
      "Waiting for deployment spec update to be observed...\n",
      "Waiting for deployment \"mymodel-mymodel-classifier-0\" rollout to finish: 0 of 1 updated replicas are available...\n",
      "Waiting for deployment spec update to be observed...\n",
      "Waiting for deployment spec update to be observed...\n",
      "Waiting for deployment \"mymodel-mymodel-classifier-0\" rollout to finish: 0 of 1 updated replicas are available...\n",
      "Waiting for deployment spec update to be observed...\n",
      "Waiting for deployment spec update to be observed...\n",
      "Waiting for deployment \"mymodel-mymodel-classifier-0\" rollout to finish: 0 of 1 updated replicas are available...\n",
      "deployment \"mymodel-mymodel-classifier-0\" successfully rolled out\n",
      "deployment \"mymodel-mymodel-svc-orch\" successfully rolled out\n"
     ]
    }
   ],
   "source": [
    "!kubectl rollout status deploy/mymodel-mymodel-classifier-0\n",
    "!kubectl rollout status deploy/mymodel-mymodel-svc-orch"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Get predictions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "from seldon_utils import *\n",
    "API_AMBASSADOR=\"localhost:8003\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### REST Request"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\n",
      "  \"meta\": {\n",
      "    \"puid\": \"hsiqcvkovr2n317f535i0jeor6\",\n",
      "    \"tags\": {\n",
      "    },\n",
      "    \"routing\": {\n",
      "    },\n",
      "    \"requestPath\": {\n",
      "      \"classifier\": \"seldonio/mock_classifier:1.0\"\n",
      "    }\n",
      "  },\n",
      "  \"data\": {\n",
      "    \"names\": [\"proba\"],\n",
      "    \"tensor\": {\n",
      "      \"shape\": [1, 1],\n",
      "      \"values\": [0.062013911980683]\n",
      "    }\n",
      "  }\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "r = rest_request_ambassador(\"mymodel\",API_AMBASSADOR)\n",
    "print(r.text)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### gRPC Request"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "meta {\n",
       "  puid: \"426s4d4ked8t7lb0efdsg3tq97\"\n",
       "  requestPath {\n",
       "    key: \"classifier\"\n",
       "    value: \"seldonio/mock_classifier:1.0\"\n",
       "  }\n",
       "}\n",
       "data {\n",
       "  names: \"proba\"\n",
       "  tensor {\n",
       "    shape: 1\n",
       "    shape: 1\n",
       "    values: 0.07317253279179549\n",
       "  }\n",
       "}"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "grpc_request_ambassador(\"mymodel\",API_AMBASSADOR)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "release \"mymodel\" deleted\r\n"
     ]
    }
   ],
   "source": [
    "!helm delete mymodel --purge"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Serve AB Test"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "NAME:   myabtest\n",
      "LAST DEPLOYED: Sun Oct 14 20:52:31 2018\n",
      "NAMESPACE: seldon\n",
      "STATUS: DEPLOYED\n",
      "\n",
      "RESOURCES:\n",
      "==> v1alpha2/SeldonDeployment\n",
      "NAME      AGE\n",
      "myabtest  0s\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "!helm install ../helm-charts/seldon-abtest --name myabtest"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[04m\u001b[31;01m-\u001b[39;49;00m\u001b[04m\u001b[31;01m-\u001b[39;49;00m\u001b[04m\u001b[31;01m-\u001b[39;49;00m\r\n",
      "\u001b[04m\u001b[31;01m#\u001b[39;49;00m \u001b[04m\u001b[31;01mS\u001b[39;49;00m\u001b[04m\u001b[31;01mo\u001b[39;49;00m\u001b[04m\u001b[31;01mu\u001b[39;49;00m\u001b[04m\u001b[31;01mr\u001b[39;49;00m\u001b[04m\u001b[31;01mc\u001b[39;49;00m\u001b[04m\u001b[31;01me\u001b[39;49;00m\u001b[04m\u001b[31;01m:\u001b[39;49;00m \u001b[04m\u001b[31;01ms\u001b[39;49;00m\u001b[04m\u001b[31;01me\u001b[39;49;00m\u001b[04m\u001b[31;01ml\u001b[39;49;00m\u001b[04m\u001b[31;01md\u001b[39;49;00m\u001b[04m\u001b[31;01mo\u001b[39;49;00m\u001b[04m\u001b[31;01mn\u001b[39;49;00m\u001b[04m\u001b[31;01m-\u001b[39;49;00m\u001b[04m\u001b[31;01ma\u001b[39;49;00m\u001b[04m\u001b[31;01mb\u001b[39;49;00m\u001b[04m\u001b[31;01mt\u001b[39;49;00m\u001b[04m\u001b[31;01me\u001b[39;49;00m\u001b[04m\u001b[31;01ms\u001b[39;49;00m\u001b[04m\u001b[31;01mt\u001b[39;49;00m\u001b[04m\u001b[31;01m/\u001b[39;49;00m\u001b[04m\u001b[31;01mt\u001b[39;49;00m\u001b[04m\u001b[31;01me\u001b[39;49;00m\u001b[04m\u001b[31;01mm\u001b[39;49;00m\u001b[04m\u001b[31;01mp\u001b[39;49;00m\u001b[04m\u001b[31;01ml\u001b[39;49;00m\u001b[04m\u001b[31;01ma\u001b[39;49;00m\u001b[04m\u001b[31;01mt\u001b[39;49;00m\u001b[04m\u001b[31;01me\u001b[39;49;00m\u001b[04m\u001b[31;01ms\u001b[39;49;00m\u001b[04m\u001b[31;01m/\u001b[39;49;00m\u001b[04m\u001b[31;01ma\u001b[39;49;00m\u001b[04m\u001b[31;01mb\u001b[39;49;00m\u001b[04m\u001b[31;01m_\u001b[39;49;00m\u001b[04m\u001b[31;01mt\u001b[39;49;00m\u001b[04m\u001b[31;01me\u001b[39;49;00m\u001b[04m\u001b[31;01ms\u001b[39;49;00m\u001b[04m\u001b[31;01mt\u001b[39;49;00m\u001b[04m\u001b[31;01m_\u001b[39;49;00m\u001b[34m1\u001b[39;49;00m\u001b[04m\u001b[31;01mp\u001b[39;49;00m\u001b[04m\u001b[31;01mo\u001b[39;49;00m\u001b[04m\u001b[31;01md\u001b[39;49;00m\u001b[04m\u001b[31;01m.\u001b[39;49;00m\u001b[04m\u001b[31;01mj\u001b[39;49;00m\u001b[04m\u001b[31;01ms\u001b[39;49;00m\u001b[04m\u001b[31;01mo\u001b[39;49;00m\u001b[04m\u001b[31;01mn\u001b[39;49;00m\r\n",
      "\r\n",
      "\r\n",
      "\u001b[04m\u001b[31;01m-\u001b[39;49;00m\u001b[04m\u001b[31;01m-\u001b[39;49;00m\u001b[04m\u001b[31;01m-\u001b[39;49;00m\r\n",
      "\u001b[04m\u001b[31;01m#\u001b[39;49;00m \u001b[04m\u001b[31;01mS\u001b[39;49;00m\u001b[04m\u001b[31;01mo\u001b[39;49;00m\u001b[04m\u001b[31;01mu\u001b[39;49;00m\u001b[04m\u001b[31;01mr\u001b[39;49;00m\u001b[04m\u001b[31;01mc\u001b[39;49;00m\u001b[04m\u001b[31;01me\u001b[39;49;00m\u001b[04m\u001b[31;01m:\u001b[39;49;00m \u001b[04m\u001b[31;01ms\u001b[39;49;00m\u001b[04m\u001b[31;01me\u001b[39;49;00m\u001b[04m\u001b[31;01ml\u001b[39;49;00m\u001b[04m\u001b[31;01md\u001b[39;49;00m\u001b[04m\u001b[31;01mo\u001b[39;49;00m\u001b[04m\u001b[31;01mn\u001b[39;49;00m\u001b[04m\u001b[31;01m-\u001b[39;49;00m\u001b[04m\u001b[31;01ma\u001b[39;49;00m\u001b[04m\u001b[31;01mb\u001b[39;49;00m\u001b[04m\u001b[31;01mt\u001b[39;49;00m\u001b[04m\u001b[31;01me\u001b[39;49;00m\u001b[04m\u001b[31;01ms\u001b[39;49;00m\u001b[04m\u001b[31;01mt\u001b[39;49;00m\u001b[04m\u001b[31;01m/\u001b[39;49;00m\u001b[04m\u001b[31;01mt\u001b[39;49;00m\u001b[04m\u001b[31;01me\u001b[39;49;00m\u001b[04m\u001b[31;01mm\u001b[39;49;00m\u001b[04m\u001b[31;01mp\u001b[39;49;00m\u001b[04m\u001b[31;01ml\u001b[39;49;00m\u001b[04m\u001b[31;01ma\u001b[39;49;00m\u001b[04m\u001b[31;01mt\u001b[39;49;00m\u001b[04m\u001b[31;01me\u001b[39;49;00m\u001b[04m\u001b[31;01ms\u001b[39;49;00m\u001b[04m\u001b[31;01m/\u001b[39;49;00m\u001b[04m\u001b[31;01ma\u001b[39;49;00m\u001b[04m\u001b[31;01mb\u001b[39;49;00m\u001b[04m\u001b[31;01m_\u001b[39;49;00m\u001b[04m\u001b[31;01mt\u001b[39;49;00m\u001b[04m\u001b[31;01me\u001b[39;49;00m\u001b[04m\u001b[31;01ms\u001b[39;49;00m\u001b[04m\u001b[31;01mt\u001b[39;49;00m\u001b[04m\u001b[31;01m_\u001b[39;49;00m\u001b[34m2\u001b[39;49;00m\u001b[04m\u001b[31;01mp\u001b[39;49;00m\u001b[04m\u001b[31;01mo\u001b[39;49;00m\u001b[04m\u001b[31;01md\u001b[39;49;00m\u001b[04m\u001b[31;01ms\u001b[39;49;00m\u001b[04m\u001b[31;01m.\u001b[39;49;00m\u001b[04m\u001b[31;01mj\u001b[39;49;00m\u001b[04m\u001b[31;01ms\u001b[39;49;00m\u001b[04m\u001b[31;01mo\u001b[39;49;00m\u001b[04m\u001b[31;01mn\u001b[39;49;00m\r\n",
      "\r\n",
      "{\r\n",
      "    \u001b[34;01m\"apiVersion\"\u001b[39;49;00m: \u001b[33m\"machinelearning.seldon.io/v1alpha2\"\u001b[39;49;00m,\r\n",
      "    \u001b[34;01m\"kind\"\u001b[39;49;00m: \u001b[33m\"SeldonDeployment\"\u001b[39;49;00m,\r\n",
      "    \u001b[34;01m\"metadata\"\u001b[39;49;00m: {\r\n",
      "\t\u001b[34;01m\"labels\"\u001b[39;49;00m: {\r\n",
      "\t    \u001b[34;01m\"app\"\u001b[39;49;00m: \u001b[33m\"seldon\"\u001b[39;49;00m\r\n",
      "\t},\r\n",
      "\t\u001b[34;01m\"name\"\u001b[39;49;00m: \u001b[33m\"myabtest\"\u001b[39;49;00m\r\n",
      "    },\r\n",
      "    \u001b[34;01m\"spec\"\u001b[39;49;00m: {\r\n",
      "\t\u001b[34;01m\"name\"\u001b[39;49;00m: \u001b[33m\"myabtest\"\u001b[39;49;00m,\r\n",
      "\t\u001b[34;01m\"predictors\"\u001b[39;49;00m: [\r\n",
      "\t    {\r\n",
      "\t\t\u001b[34;01m\"name\"\u001b[39;49;00m: \u001b[33m\"abtest\"\u001b[39;49;00m,\r\n",
      "\t\t\u001b[34;01m\"replicas\"\u001b[39;49;00m: \u001b[34m1\u001b[39;49;00m,\r\n",
      "\t\t\u001b[34;01m\"componentSpecs\"\u001b[39;49;00m: [{\r\n",
      "\t\t    \u001b[34;01m\"spec\"\u001b[39;49;00m: {\r\n",
      "\t\t\t\u001b[34;01m\"containers\"\u001b[39;49;00m: [\r\n",
      "\t\t\t    {\r\n",
      "\t\t\t\t\u001b[34;01m\"image\"\u001b[39;49;00m: \u001b[33m\"seldonio/mock_classifier:1.0\"\u001b[39;49;00m,\r\n",
      "\t\t\t\t\u001b[34;01m\"imagePullPolicy\"\u001b[39;49;00m: \u001b[33m\"IfNotPresent\"\u001b[39;49;00m,\r\n",
      "\t\t\t\t\u001b[34;01m\"name\"\u001b[39;49;00m: \u001b[33m\"classifier-1\"\u001b[39;49;00m,\r\n",
      "\t\t\t\t\u001b[34;01m\"resources\"\u001b[39;49;00m: {\r\n",
      "\t\t\t\t    \u001b[34;01m\"requests\"\u001b[39;49;00m: {\r\n",
      "\t\t\t\t\t\u001b[34;01m\"memory\"\u001b[39;49;00m: \u001b[33m\"1Mi\"\u001b[39;49;00m\r\n",
      "\t\t\t\t    }\r\n",
      "\t\t\t\t}\r\n",
      "\t\t\t    }],\r\n",
      "\t\t\t\u001b[34;01m\"terminationGracePeriodSeconds\"\u001b[39;49;00m: \u001b[34m20\u001b[39;49;00m\r\n",
      "\t\t    }},\r\n",
      "\t        {\r\n",
      "\t\t    \u001b[34;01m\"metadata\"\u001b[39;49;00m:{\r\n",
      "\t\t\t\u001b[34;01m\"labels\"\u001b[39;49;00m:{\r\n",
      "\t\t\t    \u001b[34;01m\"version\"\u001b[39;49;00m:\u001b[33m\"v2\"\u001b[39;49;00m\r\n",
      "\t\t\t}\r\n",
      "\t\t    },    \r\n",
      "\t\t\t\u001b[34;01m\"spec\"\u001b[39;49;00m:{\r\n",
      "\t\t\t    \u001b[34;01m\"containers\"\u001b[39;49;00m:[\r\n",
      "\t\t\t    {\r\n",
      "\t\t\t\t\u001b[34;01m\"image\"\u001b[39;49;00m: \u001b[33m\"seldonio/mock_classifier:1.0\"\u001b[39;49;00m,\r\n",
      "\t\t\t\t\u001b[34;01m\"imagePullPolicy\"\u001b[39;49;00m: \u001b[33m\"IfNotPresent\"\u001b[39;49;00m,\r\n",
      "\t\t\t\t\u001b[34;01m\"name\"\u001b[39;49;00m: \u001b[33m\"classifier-2\"\u001b[39;49;00m,\r\n",
      "\t\t\t\t\u001b[34;01m\"resources\"\u001b[39;49;00m: {\r\n",
      "\t\t\t\t    \u001b[34;01m\"requests\"\u001b[39;49;00m: {\r\n",
      "\t\t\t\t\t\u001b[34;01m\"memory\"\u001b[39;49;00m: \u001b[33m\"1Mi\"\u001b[39;49;00m\r\n",
      "\t\t\t\t    }\r\n",
      "\t\t\t\t}\r\n",
      "\t\t\t    }\r\n",
      "\t\t\t],\r\n",
      "\t\t\t\u001b[34;01m\"terminationGracePeriodSeconds\"\u001b[39;49;00m: \u001b[34m20\u001b[39;49;00m\r\n",
      "\t\t\t\t   }\r\n",
      "\t\t\t\t   }],\r\n",
      "\t\t\u001b[34;01m\"graph\"\u001b[39;49;00m: {\r\n",
      "\t\t    \u001b[34;01m\"name\"\u001b[39;49;00m: \u001b[33m\"myabtest\"\u001b[39;49;00m,\r\n",
      "\t\t    \u001b[34;01m\"endpoint\"\u001b[39;49;00m:{},\r\n",
      "\t\t    \u001b[34;01m\"implementation\"\u001b[39;49;00m:\u001b[33m\"RANDOM_ABTEST\"\u001b[39;49;00m,\r\n",
      "\t\t    \u001b[34;01m\"parameters\"\u001b[39;49;00m: [\r\n",
      "\t\t\t{\r\n",
      "\t\t\t    \u001b[34;01m\"name\"\u001b[39;49;00m:\u001b[33m\"ratioA\"\u001b[39;49;00m,\r\n",
      "\t\t\t    \u001b[34;01m\"value\"\u001b[39;49;00m:\u001b[33m\"0.5\"\u001b[39;49;00m,\r\n",
      "\t\t\t    \u001b[34;01m\"type\"\u001b[39;49;00m:\u001b[33m\"FLOAT\"\u001b[39;49;00m\r\n",
      "\t\t\t}\r\n",
      "\t\t    ],\r\n",
      "\t\t    \u001b[34;01m\"children\"\u001b[39;49;00m: [\r\n",
      "\t\t\t{\r\n",
      "\t\t\t    \u001b[34;01m\"name\"\u001b[39;49;00m: \u001b[33m\"classifier-1\"\u001b[39;49;00m,\r\n",
      "\t\t\t    \u001b[34;01m\"endpoint\"\u001b[39;49;00m:{\r\n",
      "\t\t\t\t\u001b[34;01m\"type\"\u001b[39;49;00m:\u001b[33m\"REST\"\u001b[39;49;00m\r\n",
      "\t\t\t    },\r\n",
      "\t\t\t    \u001b[34;01m\"type\"\u001b[39;49;00m:\u001b[33m\"MODEL\"\u001b[39;49;00m,\r\n",
      "\t\t\t    \u001b[34;01m\"children\"\u001b[39;49;00m:[]\r\n",
      "\t\t\t},\r\n",
      "\t\t\t{\r\n",
      "\t\t\t    \u001b[34;01m\"name\"\u001b[39;49;00m: \u001b[33m\"classifier-2\"\u001b[39;49;00m,\r\n",
      "\t\t\t    \u001b[34;01m\"endpoint\"\u001b[39;49;00m:{\r\n",
      "\t\t\t\t\u001b[34;01m\"type\"\u001b[39;49;00m:\u001b[33m\"REST\"\u001b[39;49;00m\r\n",
      "\t\t\t    },\r\n",
      "\t\t\t    \u001b[34;01m\"type\"\u001b[39;49;00m:\u001b[33m\"MODEL\"\u001b[39;49;00m,\r\n",
      "\t\t\t    \u001b[34;01m\"children\"\u001b[39;49;00m:[]\r\n",
      "\t\t\t}   \r\n",
      "\t\t    ]\r\n",
      "\t\t}\r\n",
      "\t    }\r\n",
      "\t]\r\n",
      "    }\r\n",
      "}\t\t\r\n",
      "\t\t\r\n",
      "\r\n",
      " \r\n"
     ]
    }
   ],
   "source": [
    "!helm template ../helm-charts/seldon-abtest --name myabtest | pygmentize -l json"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "deployment \"myabtest-abtest-classifier-1-0\" successfully rolled out\n",
      "deployment \"myabtest-abtest-classifier-1-0\" successfully rolled out\n",
      "Waiting for deployment \"myabtest-abtest-svc-orch\" rollout to finish: 0 of 1 updated replicas are available...\n",
      "deployment \"myabtest-abtest-svc-orch\" successfully rolled out\n"
     ]
    }
   ],
   "source": [
    "!kubectl rollout status deploy/myabtest-abtest-classifier-1-0\n",
    "!kubectl rollout status deploy/myabtest-abtest-classifier-1-0\n",
    "!kubectl rollout status deploy/myabtest-abtest-svc-orch "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Get predictions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "from seldon_utils import *\n",
    "API_AMBASSADOR=\"localhost:8003\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### REST Request"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\n",
      "  \"meta\": {\n",
      "    \"puid\": \"romhq0ortlbhcosq08nle1dnj6\",\n",
      "    \"tags\": {\n",
      "    },\n",
      "    \"routing\": {\n",
      "      \"myabtest\": 1\n",
      "    },\n",
      "    \"requestPath\": {\n",
      "      \"myabtest\": \"\",\n",
      "      \"classifier-2\": \"seldonio/mock_classifier:1.0\"\n",
      "    }\n",
      "  },\n",
      "  \"data\": {\n",
      "    \"names\": [\"proba\"],\n",
      "    \"tensor\": {\n",
      "      \"shape\": [1, 1],\n",
      "      \"values\": [0.06798732540491603]\n",
      "    }\n",
      "  }\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "r = rest_request_ambassador(\"myabtest\",API_AMBASSADOR)\n",
    "print(r.text)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### gRPC Request"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "meta {\n",
       "  puid: \"ugv38o8aq1pkl38nvf0m1tv4c8\"\n",
       "  routing {\n",
       "    key: \"myabtest\"\n",
       "  }\n",
       "  requestPath {\n",
       "    key: \"classifier-1\"\n",
       "    value: \"seldonio/mock_classifier:1.0\"\n",
       "  }\n",
       "  requestPath {\n",
       "    key: \"myabtest\"\n",
       "  }\n",
       "}\n",
       "data {\n",
       "  names: \"proba\"\n",
       "  tensor {\n",
       "    shape: 1\n",
       "    shape: 1\n",
       "    values: 0.07245218440685676\n",
       "  }\n",
       "}"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "grpc_request_ambassador(\"myabtest\",API_AMBASSADOR)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "release \"myabtest\" deleted\r\n"
     ]
    }
   ],
   "source": [
    "!helm delete myabtest --purge"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Serve Multi-Armed Bandit"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "NAME:   mymab\n",
      "LAST DEPLOYED: Sun Oct 14 20:52:58 2018\n",
      "NAMESPACE: seldon\n",
      "STATUS: DEPLOYED\n",
      "\n",
      "RESOURCES:\n",
      "==> v1alpha2/SeldonDeployment\n",
      "NAME   AGE\n",
      "mymab  0s\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "!helm install ../helm-charts/seldon-mab --name mymab"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[04m\u001b[31;01m-\u001b[39;49;00m\u001b[04m\u001b[31;01m-\u001b[39;49;00m\u001b[04m\u001b[31;01m-\u001b[39;49;00m\r\n",
      "\u001b[04m\u001b[31;01m#\u001b[39;49;00m \u001b[04m\u001b[31;01mS\u001b[39;49;00m\u001b[04m\u001b[31;01mo\u001b[39;49;00m\u001b[04m\u001b[31;01mu\u001b[39;49;00m\u001b[04m\u001b[31;01mr\u001b[39;49;00m\u001b[04m\u001b[31;01mc\u001b[39;49;00m\u001b[04m\u001b[31;01me\u001b[39;49;00m\u001b[04m\u001b[31;01m:\u001b[39;49;00m \u001b[04m\u001b[31;01ms\u001b[39;49;00m\u001b[04m\u001b[31;01me\u001b[39;49;00m\u001b[04m\u001b[31;01ml\u001b[39;49;00m\u001b[04m\u001b[31;01md\u001b[39;49;00m\u001b[04m\u001b[31;01mo\u001b[39;49;00m\u001b[04m\u001b[31;01mn\u001b[39;49;00m\u001b[04m\u001b[31;01m-\u001b[39;49;00m\u001b[04m\u001b[31;01mm\u001b[39;49;00m\u001b[04m\u001b[31;01ma\u001b[39;49;00m\u001b[04m\u001b[31;01mb\u001b[39;49;00m\u001b[04m\u001b[31;01m/\u001b[39;49;00m\u001b[04m\u001b[31;01mt\u001b[39;49;00m\u001b[04m\u001b[31;01me\u001b[39;49;00m\u001b[04m\u001b[31;01mm\u001b[39;49;00m\u001b[04m\u001b[31;01mp\u001b[39;49;00m\u001b[04m\u001b[31;01ml\u001b[39;49;00m\u001b[04m\u001b[31;01ma\u001b[39;49;00m\u001b[04m\u001b[31;01mt\u001b[39;49;00m\u001b[04m\u001b[31;01me\u001b[39;49;00m\u001b[04m\u001b[31;01ms\u001b[39;49;00m\u001b[04m\u001b[31;01m/\u001b[39;49;00m\u001b[04m\u001b[31;01mm\u001b[39;49;00m\u001b[04m\u001b[31;01ma\u001b[39;49;00m\u001b[04m\u001b[31;01mb\u001b[39;49;00m\u001b[04m\u001b[31;01m.\u001b[39;49;00m\u001b[04m\u001b[31;01mj\u001b[39;49;00m\u001b[04m\u001b[31;01ms\u001b[39;49;00m\u001b[04m\u001b[31;01mo\u001b[39;49;00m\u001b[04m\u001b[31;01mn\u001b[39;49;00m\r\n",
      "{\r\n",
      "    \u001b[34;01m\"apiVersion\"\u001b[39;49;00m: \u001b[33m\"machinelearning.seldon.io/v1alpha2\"\u001b[39;49;00m,\r\n",
      "    \u001b[34;01m\"kind\"\u001b[39;49;00m: \u001b[33m\"SeldonDeployment\"\u001b[39;49;00m,\r\n",
      "    \u001b[34;01m\"metadata\"\u001b[39;49;00m: {\r\n",
      "\t\u001b[34;01m\"labels\"\u001b[39;49;00m: {\r\n",
      "\t    \u001b[34;01m\"app\"\u001b[39;49;00m: \u001b[33m\"seldon\"\u001b[39;49;00m\r\n",
      "\t},\r\n",
      "\t\u001b[34;01m\"name\"\u001b[39;49;00m: \u001b[33m\"mymab\"\u001b[39;49;00m\r\n",
      "    },\r\n",
      "    \u001b[34;01m\"spec\"\u001b[39;49;00m: {\r\n",
      "\t\u001b[34;01m\"name\"\u001b[39;49;00m: \u001b[33m\"mymab\"\u001b[39;49;00m,\r\n",
      "\t\u001b[34;01m\"predictors\"\u001b[39;49;00m: [\r\n",
      "\t    {\r\n",
      "\t\t\u001b[34;01m\"name\"\u001b[39;49;00m: \u001b[33m\"abtest\"\u001b[39;49;00m,\r\n",
      "\t\t\u001b[34;01m\"replicas\"\u001b[39;49;00m: \u001b[34m1\u001b[39;49;00m,\r\n",
      "\t\t\u001b[34;01m\"componentSpecs\"\u001b[39;49;00m: [{\r\n",
      "\t\t    \u001b[34;01m\"spec\"\u001b[39;49;00m: {\r\n",
      "\t\t\t\u001b[34;01m\"containers\"\u001b[39;49;00m: [\r\n",
      "\t\t\t    {\r\n",
      "\t\t\t\t\u001b[34;01m\"image\"\u001b[39;49;00m: \u001b[33m\"seldonio/mock_classifier:1.0\"\u001b[39;49;00m,\r\n",
      "\t\t\t\t\u001b[34;01m\"imagePullPolicy\"\u001b[39;49;00m: \u001b[33m\"IfNotPresent\"\u001b[39;49;00m,\r\n",
      "\t\t\t\t\u001b[34;01m\"name\"\u001b[39;49;00m: \u001b[33m\"classifier-1\"\u001b[39;49;00m,\r\n",
      "\t\t\t\t\u001b[34;01m\"resources\"\u001b[39;49;00m: {\r\n",
      "\t\t\t\t    \u001b[34;01m\"requests\"\u001b[39;49;00m: {\r\n",
      "\t\t\t\t\t\u001b[34;01m\"memory\"\u001b[39;49;00m: \u001b[33m\"1Mi\"\u001b[39;49;00m\r\n",
      "\t\t\t\t    }\r\n",
      "\t\t\t\t}\r\n",
      "\t\t\t    }],\r\n",
      "\t\t\t\u001b[34;01m\"terminationGracePeriodSeconds\"\u001b[39;49;00m: \u001b[34m20\u001b[39;49;00m\r\n",
      "\t\t    }},\r\n",
      "\t        {\r\n",
      "\t\t    \u001b[34;01m\"metadata\"\u001b[39;49;00m:{\r\n",
      "\t\t\t\u001b[34;01m\"labels\"\u001b[39;49;00m:{\r\n",
      "\t\t\t    \u001b[34;01m\"version\"\u001b[39;49;00m:\u001b[33m\"v2\"\u001b[39;49;00m\r\n",
      "\t\t\t}\r\n",
      "\t\t    },    \r\n",
      "\t\t\t\u001b[34;01m\"spec\"\u001b[39;49;00m:{\r\n",
      "\t\t\t    \u001b[34;01m\"containers\"\u001b[39;49;00m:[\r\n",
      "\t\t\t    {\r\n",
      "\t\t\t\t\u001b[34;01m\"image\"\u001b[39;49;00m: \u001b[33m\"seldonio/mock_classifier:1.0\"\u001b[39;49;00m,\r\n",
      "\t\t\t\t\u001b[34;01m\"imagePullPolicy\"\u001b[39;49;00m: \u001b[33m\"IfNotPresent\"\u001b[39;49;00m,\r\n",
      "\t\t\t\t\u001b[34;01m\"name\"\u001b[39;49;00m: \u001b[33m\"classifier-2\"\u001b[39;49;00m,\r\n",
      "\t\t\t\t\u001b[34;01m\"resources\"\u001b[39;49;00m: {\r\n",
      "\t\t\t\t    \u001b[34;01m\"requests\"\u001b[39;49;00m: {\r\n",
      "\t\t\t\t\t\u001b[34;01m\"memory\"\u001b[39;49;00m: \u001b[33m\"1Mi\"\u001b[39;49;00m\r\n",
      "\t\t\t\t    }\r\n",
      "\t\t\t\t}\r\n",
      "\t\t\t    }\r\n",
      "\t\t\t],\r\n",
      "\t\t\t\u001b[34;01m\"terminationGracePeriodSeconds\"\u001b[39;49;00m: \u001b[34m20\u001b[39;49;00m\r\n",
      "\t\t\t}\r\n",
      "\t\t},\r\n",
      "\t        {\r\n",
      "\t\t    \u001b[34;01m\"spec\"\u001b[39;49;00m:{\r\n",
      "\t\t\t\u001b[34;01m\"containers\"\u001b[39;49;00m: [{\r\n",
      "\t\t\t    \u001b[34;01m\"image\"\u001b[39;49;00m: \u001b[33m\"seldonio/mab_epsilon_greedy:1.1\"\u001b[39;49;00m,\r\n",
      "\t\t\t    \u001b[34;01m\"name\"\u001b[39;49;00m: \u001b[33m\"eg-router\"\u001b[39;49;00m\r\n",
      "\t\t\t}],\r\n",
      "\t\t\t\u001b[34;01m\"terminationGracePeriodSeconds\"\u001b[39;49;00m: \u001b[34m20\u001b[39;49;00m\r\n",
      "\t\t    }}\r\n",
      "\t        ],\r\n",
      "\t\t\u001b[34;01m\"graph\"\u001b[39;49;00m: {\r\n",
      "\t\t    \u001b[34;01m\"name\"\u001b[39;49;00m: \u001b[33m\"eg-router\"\u001b[39;49;00m,\r\n",
      "\t\t    \u001b[34;01m\"type\"\u001b[39;49;00m:\u001b[33m\"ROUTER\"\u001b[39;49;00m,\r\n",
      "\t\t    \u001b[34;01m\"parameters\"\u001b[39;49;00m: [\r\n",
      "\t\t\t{\r\n",
      "\t\t\t    \u001b[34;01m\"name\"\u001b[39;49;00m: \u001b[33m\"n_branches\"\u001b[39;49;00m,\r\n",
      "\t\t\t    \u001b[34;01m\"value\"\u001b[39;49;00m: \u001b[33m\"2\"\u001b[39;49;00m,\r\n",
      "\t\t\t    \u001b[34;01m\"type\"\u001b[39;49;00m: \u001b[33m\"INT\"\u001b[39;49;00m\r\n",
      "\t\t\t},\r\n",
      "\t\t\t{\r\n",
      "\t\t\t    \u001b[34;01m\"name\"\u001b[39;49;00m: \u001b[33m\"epsilon\"\u001b[39;49;00m,\r\n",
      "\t\t\t    \u001b[34;01m\"value\"\u001b[39;49;00m: \u001b[33m\"0.2\"\u001b[39;49;00m,\r\n",
      "\t\t\t    \u001b[34;01m\"type\"\u001b[39;49;00m: \u001b[33m\"FLOAT\"\u001b[39;49;00m\r\n",
      "\t\t\t},\r\n",
      "\t\t\t{\r\n",
      "\t\t\t    \u001b[34;01m\"name\"\u001b[39;49;00m: \u001b[33m\"verbose\"\u001b[39;49;00m,\r\n",
      "\t\t\t    \u001b[34;01m\"value\"\u001b[39;49;00m: \u001b[33m\"1\"\u001b[39;49;00m,\r\n",
      "\t\t\t    \u001b[34;01m\"type\"\u001b[39;49;00m: \u001b[33m\"BOOL\"\u001b[39;49;00m\r\n",
      "\t\t\t}\r\n",
      "\t\t    ],\r\n",
      "\t\t    \u001b[34;01m\"children\"\u001b[39;49;00m: [\r\n",
      "\t\t\t{\r\n",
      "\t\t\t    \u001b[34;01m\"name\"\u001b[39;49;00m: \u001b[33m\"classifier-1\"\u001b[39;49;00m,\r\n",
      "\t\t\t    \u001b[34;01m\"endpoint\"\u001b[39;49;00m:{\r\n",
      "\t\t\t\t\u001b[34;01m\"type\"\u001b[39;49;00m:\u001b[33m\"REST\"\u001b[39;49;00m\r\n",
      "\t\t\t    },\r\n",
      "\t\t\t    \u001b[34;01m\"type\"\u001b[39;49;00m:\u001b[33m\"MODEL\"\u001b[39;49;00m,\r\n",
      "\t\t\t    \u001b[34;01m\"children\"\u001b[39;49;00m:[]\r\n",
      "\t\t\t},\r\n",
      "\t\t\t{\r\n",
      "\t\t\t    \u001b[34;01m\"name\"\u001b[39;49;00m: \u001b[33m\"classifier-2\"\u001b[39;49;00m,\r\n",
      "\t\t\t    \u001b[34;01m\"endpoint\"\u001b[39;49;00m:{\r\n",
      "\t\t\t\t\u001b[34;01m\"type\"\u001b[39;49;00m:\u001b[33m\"REST\"\u001b[39;49;00m\r\n",
      "\t\t\t    },\r\n",
      "\t\t\t    \u001b[34;01m\"type\"\u001b[39;49;00m:\u001b[33m\"MODEL\"\u001b[39;49;00m,\r\n",
      "\t\t\t    \u001b[34;01m\"children\"\u001b[39;49;00m:[]\r\n",
      "\t\t\t}   \r\n",
      "\t\t    ]\r\n",
      "\t\t}\r\n",
      "\t    }\r\n",
      "\t]\r\n",
      "    }\r\n",
      "}\r\n",
      "\t\t\r\n",
      "\r\n",
      " \r\n"
     ]
    }
   ],
   "source": [
    "!helm template ../helm-charts/seldon-mab --name mymab | pygmentize -l json"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Waiting for deployment \"mymab-abtest-classifier-1-0\" rollout to finish: 0 of 1 updated replicas are available...\n",
      "deployment \"mymab-abtest-classifier-1-0\" successfully rolled out\n",
      "Waiting for deployment \"mymab-abtest-classifier-2-1\" rollout to finish: 0 of 1 updated replicas are available...\n",
      "deployment \"mymab-abtest-classifier-2-1\" successfully rolled out\n",
      "Waiting for deployment \"mymab-abtest-eg-router-2\" rollout to finish: 0 of 1 updated replicas are available...\n",
      "Waiting for deployment spec update to be observed...\n",
      "Waiting for deployment spec update to be observed...\n",
      "Waiting for deployment \"mymab-abtest-eg-router-2\" rollout to finish: 0 of 1 updated replicas are available...\n",
      "Waiting for deployment spec update to be observed...\n",
      "Waiting for deployment spec update to be observed...\n",
      "Waiting for deployment \"mymab-abtest-eg-router-2\" rollout to finish: 0 of 1 updated replicas are available...\n",
      "Waiting for deployment spec update to be observed...\n",
      "Waiting for deployment spec update to be observed...\n",
      "Waiting for deployment \"mymab-abtest-eg-router-2\" rollout to finish: 0 of 1 updated replicas are available...\n",
      "Waiting for deployment spec update to be observed...\n",
      "Waiting for deployment spec update to be observed...\n",
      "Waiting for deployment \"mymab-abtest-eg-router-2\" rollout to finish: 0 of 1 updated replicas are available...\n",
      "Waiting for deployment spec update to be observed...\n",
      "Waiting for deployment spec update to be observed...\n",
      "Waiting for deployment \"mymab-abtest-eg-router-2\" rollout to finish: 0 of 1 updated replicas are available...\n",
      "Waiting for deployment spec update to be observed...\n",
      "Waiting for deployment spec update to be observed...\n",
      "Waiting for deployment spec update to be observed...\n",
      "Waiting for deployment spec update to be observed...\n",
      "Waiting for deployment spec update to be observed...\n",
      "Waiting for deployment spec update to be observed...\n",
      "Waiting for deployment \"mymab-abtest-eg-router-2\" rollout to finish: 0 of 1 updated replicas are available...\n",
      "deployment \"mymab-abtest-eg-router-2\" successfully rolled out\n"
     ]
    }
   ],
   "source": [
    "!kubectl rollout status deploy/mymab-abtest-classifier-1-0\n",
    "!kubectl rollout status deploy/mymab-abtest-classifier-2-1\n",
    "!kubectl rollout status deploy/mymab-abtest-eg-router-2 "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Get predictions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [],
   "source": [
    "from seldon_utils import *\n",
    "API_AMBASSADOR=\"localhost:8003\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### REST Request"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\n",
      "  \"meta\": {\n",
      "    \"puid\": \"l6nll08e5msr8s39cpumsptokr\",\n",
      "    \"tags\": {\n",
      "    },\n",
      "    \"routing\": {\n",
      "      \"eg-router\": 0\n",
      "    },\n",
      "    \"requestPath\": {\n",
      "      \"eg-router\": \"seldonio/mab_epsilon_greedy:1.1\",\n",
      "      \"classifier-1\": \"seldonio/mock_classifier:1.0\"\n",
      "    }\n",
      "  },\n",
      "  \"data\": {\n",
      "    \"names\": [\"proba\"],\n",
      "    \"tensor\": {\n",
      "      \"shape\": [1, 1],\n",
      "      \"values\": [0.08857813159319305]\n",
      "    }\n",
      "  }\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "r = rest_request_ambassador(\"mymab\",API_AMBASSADOR)\n",
    "print(r.text)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### gRPC Request"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "meta {\n",
       "  puid: \"ompco5g59k1jj2svr1db7h3djm\"\n",
       "  routing {\n",
       "    key: \"eg-router\"\n",
       "  }\n",
       "  requestPath {\n",
       "    key: \"classifier-1\"\n",
       "    value: \"seldonio/mock_classifier:1.0\"\n",
       "  }\n",
       "  requestPath {\n",
       "    key: \"eg-router\"\n",
       "    value: \"seldonio/mab_epsilon_greedy:1.1\"\n",
       "  }\n",
       "}\n",
       "data {\n",
       "  names: \"proba\"\n",
       "  tensor {\n",
       "    shape: 1\n",
       "    shape: 1\n",
       "    values: 0.08072007852083082\n",
       "  }\n",
       "}"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "grpc_request_ambassador(\"mymab\",API_AMBASSADOR)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "release \"mymab\" deleted\r\n"
     ]
    }
   ],
   "source": [
    "!helm delete mymab --purge"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Serve Single Model with Outlier Detector"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "NAME:   mymodel\r\n",
      "LAST DEPLOYED: Sun Oct 14 20:53:57 2018\r\n",
      "NAMESPACE: seldon\r\n",
      "STATUS: DEPLOYED\r\n",
      "\r\n",
      "RESOURCES:\r\n",
      "==> v1alpha2/SeldonDeployment\r\n",
      "NAME     AGE\r\n",
      "mymodel  0s\r\n",
      "\r\n",
      "\r\n"
     ]
    }
   ],
   "source": [
    "!helm install ../helm-charts/seldon-single-model --name mymodel --set outlier_detection.enabled=true "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[04m\u001b[31;01m-\u001b[39;49;00m\u001b[04m\u001b[31;01m-\u001b[39;49;00m\u001b[04m\u001b[31;01m-\u001b[39;49;00m\r\n",
      "\u001b[04m\u001b[31;01m#\u001b[39;49;00m \u001b[04m\u001b[31;01mS\u001b[39;49;00m\u001b[04m\u001b[31;01mo\u001b[39;49;00m\u001b[04m\u001b[31;01mu\u001b[39;49;00m\u001b[04m\u001b[31;01mr\u001b[39;49;00m\u001b[04m\u001b[31;01mc\u001b[39;49;00m\u001b[04m\u001b[31;01me\u001b[39;49;00m\u001b[04m\u001b[31;01m:\u001b[39;49;00m \u001b[04m\u001b[31;01ms\u001b[39;49;00m\u001b[04m\u001b[31;01me\u001b[39;49;00m\u001b[04m\u001b[31;01ml\u001b[39;49;00m\u001b[04m\u001b[31;01md\u001b[39;49;00m\u001b[04m\u001b[31;01mo\u001b[39;49;00m\u001b[04m\u001b[31;01mn\u001b[39;49;00m\u001b[04m\u001b[31;01m-\u001b[39;49;00m\u001b[04m\u001b[31;01ms\u001b[39;49;00m\u001b[04m\u001b[31;01mi\u001b[39;49;00m\u001b[04m\u001b[31;01mn\u001b[39;49;00m\u001b[04m\u001b[31;01mg\u001b[39;49;00m\u001b[04m\u001b[31;01ml\u001b[39;49;00m\u001b[04m\u001b[31;01me\u001b[39;49;00m\u001b[04m\u001b[31;01m-\u001b[39;49;00m\u001b[04m\u001b[31;01mm\u001b[39;49;00m\u001b[04m\u001b[31;01mo\u001b[39;49;00m\u001b[04m\u001b[31;01md\u001b[39;49;00m\u001b[04m\u001b[31;01me\u001b[39;49;00m\u001b[04m\u001b[31;01ml\u001b[39;49;00m\u001b[04m\u001b[31;01m/\u001b[39;49;00m\u001b[04m\u001b[31;01mt\u001b[39;49;00m\u001b[04m\u001b[31;01me\u001b[39;49;00m\u001b[04m\u001b[31;01mm\u001b[39;49;00m\u001b[04m\u001b[31;01mp\u001b[39;49;00m\u001b[04m\u001b[31;01ml\u001b[39;49;00m\u001b[04m\u001b[31;01ma\u001b[39;49;00m\u001b[04m\u001b[31;01mt\u001b[39;49;00m\u001b[04m\u001b[31;01me\u001b[39;49;00m\u001b[04m\u001b[31;01ms\u001b[39;49;00m\u001b[04m\u001b[31;01m/\u001b[39;49;00m\u001b[04m\u001b[31;01mm\u001b[39;49;00m\u001b[04m\u001b[31;01mo\u001b[39;49;00m\u001b[04m\u001b[31;01md\u001b[39;49;00m\u001b[04m\u001b[31;01me\u001b[39;49;00m\u001b[04m\u001b[31;01ml\u001b[39;49;00m\u001b[04m\u001b[31;01m.\u001b[39;49;00m\u001b[04m\u001b[31;01mj\u001b[39;49;00m\u001b[04m\u001b[31;01ms\u001b[39;49;00m\u001b[04m\u001b[31;01mo\u001b[39;49;00m\u001b[04m\u001b[31;01mn\u001b[39;49;00m\r\n",
      "{\r\n",
      "    \u001b[34;01m\"apiVersion\"\u001b[39;49;00m: \u001b[33m\"machinelearning.seldon.io/v1alpha2\"\u001b[39;49;00m,\r\n",
      "    \u001b[34;01m\"kind\"\u001b[39;49;00m: \u001b[33m\"SeldonDeployment\"\u001b[39;49;00m,\r\n",
      "    \u001b[34;01m\"metadata\"\u001b[39;49;00m: {\r\n",
      "        \u001b[34;01m\"labels\"\u001b[39;49;00m: {\r\n",
      "            \u001b[34;01m\"app\"\u001b[39;49;00m: \u001b[33m\"seldon\"\u001b[39;49;00m\r\n",
      "        },\r\n",
      "        \u001b[34;01m\"name\"\u001b[39;49;00m: \u001b[33m\"mymodel\"\u001b[39;49;00m\r\n",
      "    },\r\n",
      "    \u001b[34;01m\"spec\"\u001b[39;49;00m: {\r\n",
      "        \u001b[34;01m\"name\"\u001b[39;49;00m: \u001b[33m\"mymodel\"\u001b[39;49;00m,\r\n",
      "        \u001b[34;01m\"predictors\"\u001b[39;49;00m: [\r\n",
      "            {\r\n",
      "                \u001b[34;01m\"componentSpecs\"\u001b[39;49;00m: [{\r\n",
      "                    \u001b[34;01m\"spec\"\u001b[39;49;00m: {\r\n",
      "                        \u001b[34;01m\"containers\"\u001b[39;49;00m: [\r\n",
      "                            {\r\n",
      "                                \u001b[34;01m\"image\"\u001b[39;49;00m: \u001b[33m\"seldonio/mock_classifier:1.0\"\u001b[39;49;00m,\r\n",
      "                                \u001b[34;01m\"imagePullPolicy\"\u001b[39;49;00m: \u001b[33m\"IfNotPresent\"\u001b[39;49;00m,\r\n",
      "                                \u001b[34;01m\"name\"\u001b[39;49;00m: \u001b[33m\"classifier\"\u001b[39;49;00m,\r\n",
      "                                \u001b[34;01m\"resources\"\u001b[39;49;00m: {\r\n",
      "                                    \u001b[34;01m\"requests\"\u001b[39;49;00m: {\r\n",
      "                                        \u001b[34;01m\"memory\"\u001b[39;49;00m: \u001b[33m\"1Mi\"\u001b[39;49;00m\r\n",
      "                                    }\r\n",
      "                                }\r\n",
      "                            }\r\n",
      "                        ],\r\n",
      "                        \u001b[34;01m\"terminationGracePeriodSeconds\"\u001b[39;49;00m: \u001b[34m1\u001b[39;49;00m\r\n",
      "                    }}        \t\t\t\t   \r\n",
      "\t\t    ,\r\n",
      "\t\t    {\r\n",
      "\t\t    \u001b[34;01m\"spec\"\u001b[39;49;00m: {\r\n",
      "\t\t\t\u001b[34;01m\"containers\"\u001b[39;49;00m: [\r\n",
      "\t\t\t    {\r\n",
      "\t\t\t\t\u001b[34;01m\"image\"\u001b[39;49;00m: \u001b[33m\"seldonio/outlier_mahalanobis:0.3\"\u001b[39;49;00m,\r\n",
      "\t\t\t\t\u001b[34;01m\"imagePullPolicy\"\u001b[39;49;00m: \u001b[33m\"IfNotPresent\"\u001b[39;49;00m,\r\n",
      "\t\t\t\t\u001b[34;01m\"name\"\u001b[39;49;00m: \u001b[33m\"outlier-detector\"\u001b[39;49;00m,\r\n",
      "\t\t\t\t\u001b[34;01m\"resources\"\u001b[39;49;00m: {\r\n",
      "\t\t\t\t    \u001b[34;01m\"requests\"\u001b[39;49;00m: {\r\n",
      "\t\t\t\t\t\u001b[34;01m\"memory\"\u001b[39;49;00m: \u001b[33m\"1Mi\"\u001b[39;49;00m\r\n",
      "\t\t\t\t    }\r\n",
      "\t\t\t\t}\r\n",
      "\t\t\t    }\r\n",
      "\t\t\t],\r\n",
      "\t\t\t\u001b[34;01m\"terminationGracePeriodSeconds\"\u001b[39;49;00m: \u001b[34m20\u001b[39;49;00m\r\n",
      "\t\t    }\r\n",
      "\t\t    }\t\t\t\t   \r\n",
      "\t        ],\r\n",
      "                \u001b[34;01m\"graph\"\u001b[39;49;00m:\r\n",
      "\t\t {\r\n",
      "\t\t    \u001b[34;01m\"name\"\u001b[39;49;00m: \u001b[33m\"outlier-detector\"\u001b[39;49;00m,\r\n",
      "\t\t    \u001b[34;01m\"type\"\u001b[39;49;00m: \u001b[33m\"TRANSFORMER\"\u001b[39;49;00m,\r\n",
      "\t\t    \u001b[34;01m\"parameters\"\u001b[39;49;00m: [\r\n",
      "\t\t\t{\r\n",
      "\t\t\t    \u001b[34;01m\"name\"\u001b[39;49;00m: \u001b[33m\"n_components\"\u001b[39;49;00m,\r\n",
      "\t\t\t    \u001b[34;01m\"value\"\u001b[39;49;00m: \u001b[33m\"2\"\u001b[39;49;00m,\r\n",
      "\t\t\t    \u001b[34;01m\"type\"\u001b[39;49;00m: \u001b[33m\"INT\"\u001b[39;49;00m\r\n",
      "\t\t\t}\r\n",
      "\t\t    ],\r\n",
      "\t\t    \u001b[34;01m\"endpoint\"\u001b[39;49;00m: {\r\n",
      "\t\t\t\u001b[34;01m\"type\"\u001b[39;49;00m: \u001b[33m\"REST\"\u001b[39;49;00m\r\n",
      "\t\t    },\r\n",
      "\t\t     \u001b[34;01m\"children\"\u001b[39;49;00m: [\t\t\t \r\n",
      "\t\t{\r\n",
      "                    \u001b[34;01m\"children\"\u001b[39;49;00m: [],\r\n",
      "                    \u001b[34;01m\"name\"\u001b[39;49;00m: \u001b[33m\"classifier\"\u001b[39;49;00m,\r\n",
      "                    \u001b[34;01m\"endpoint\"\u001b[39;49;00m: {\r\n",
      "\t\t\t\u001b[34;01m\"type\"\u001b[39;49;00m : \u001b[33m\"REST\"\u001b[39;49;00m\r\n",
      "\t\t    },\r\n",
      "                    \u001b[34;01m\"type\"\u001b[39;49;00m: \u001b[33m\"MODEL\"\u001b[39;49;00m\r\n",
      "                }\r\n",
      "\t\t     ]},\t\t\t \t\t\t \r\n",
      "                \u001b[34;01m\"name\"\u001b[39;49;00m: \u001b[33m\"mymodel\"\u001b[39;49;00m,\r\n",
      "                \u001b[34;01m\"replicas\"\u001b[39;49;00m: \u001b[34m1\u001b[39;49;00m,\r\n",
      "\t\t\u001b[34;01m\"labels\"\u001b[39;49;00m: {\r\n",
      "\t\t    \u001b[34;01m\"version\"\u001b[39;49;00m : \u001b[33m\"v1\"\u001b[39;49;00m\r\n",
      "\t\t}\r\n",
      "            }\r\n",
      "        ]\r\n",
      "    }\r\n",
      "}\r\n"
     ]
    }
   ],
   "source": [
    "!helm template ../helm-charts/seldon-single-model --name mymodel --set outlier_detection.enabled=true | pygmentize -l json"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Waiting for deployment \"mymodel-mymodel-classifier-0\" rollout to finish: 0 of 1 updated replicas are available...\n",
      "deployment \"mymodel-mymodel-classifier-0\" successfully rolled out\n",
      "Waiting for deployment \"mymodel-mymodel-svc-orch\" rollout to finish: 0 of 1 updated replicas are available...\n",
      "Waiting for deployment spec update to be observed...\n",
      "Waiting for deployment spec update to be observed...\n",
      "Waiting for deployment \"mymodel-mymodel-svc-orch\" rollout to finish: 0 of 1 updated replicas are available...\n",
      "deployment \"mymodel-mymodel-svc-orch\" successfully rolled out\n",
      "Waiting for deployment \"mymodel-mymodel-outlier-detector-1\" rollout to finish: 0 of 1 updated replicas are available...\n",
      "Waiting for deployment spec update to be observed...\n",
      "Waiting for deployment spec update to be observed...\n",
      "Waiting for deployment \"mymodel-mymodel-outlier-detector-1\" rollout to finish: 0 of 1 updated replicas are available...\n",
      "Waiting for deployment spec update to be observed...\n",
      "Waiting for deployment spec update to be observed...\n",
      "Waiting for deployment \"mymodel-mymodel-outlier-detector-1\" rollout to finish: 0 of 1 updated replicas are available...\n",
      "Waiting for deployment spec update to be observed...\n",
      "Waiting for deployment spec update to be observed...\n",
      "Waiting for deployment \"mymodel-mymodel-outlier-detector-1\" rollout to finish: 0 of 1 updated replicas are available...\n",
      "Waiting for deployment spec update to be observed...\n",
      "Waiting for deployment spec update to be observed...\n",
      "Waiting for deployment \"mymodel-mymodel-outlier-detector-1\" rollout to finish: 0 of 1 updated replicas are available...\n",
      "deployment \"mymodel-mymodel-outlier-detector-1\" successfully rolled out\n"
     ]
    }
   ],
   "source": [
    "!kubectl rollout status deploy/mymodel-mymodel-classifier-0\n",
    "!kubectl rollout status deploy/mymodel-mymodel-svc-orch\n",
    "!kubectl rollout status deploy/mymodel-mymodel-outlier-detector-1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Get predictions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [],
   "source": [
    "from seldon_utils import *\n",
    "API_AMBASSADOR=\"localhost:8003\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### REST Request"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\n",
      "  \"meta\": {\n",
      "    \"puid\": \"7e04p8pfklgnnimj3vc794tr6k\",\n",
      "    \"tags\": {\n",
      "      \"outlierScore\": [0.0]\n",
      "    },\n",
      "    \"routing\": {\n",
      "      \"outlier-detector\": -1\n",
      "    },\n",
      "    \"requestPath\": {\n",
      "      \"outlier-detector\": \"seldonio/outlier_mahalanobis:0.3\",\n",
      "      \"classifier\": \"seldonio/mock_classifier:1.0\"\n",
      "    }\n",
      "  },\n",
      "  \"data\": {\n",
      "    \"names\": [\"proba\"],\n",
      "    \"tensor\": {\n",
      "      \"shape\": [1, 1],\n",
      "      \"values\": [0.08955540764238194]\n",
      "    }\n",
      "  }\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "r = rest_request_ambassador(\"mymodel\",API_AMBASSADOR)\n",
    "print(r.text)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### gRPC Request"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "meta {\n",
       "  puid: \"pjil5hcqmh6m7ar1rmu2db5abi\"\n",
       "  tags {\n",
       "    key: \"outlierScore\"\n",
       "    value {\n",
       "      list_value {\n",
       "        values {\n",
       "          number_value: 0.0\n",
       "        }\n",
       "      }\n",
       "    }\n",
       "  }\n",
       "  routing {\n",
       "    key: \"outlier-detector\"\n",
       "    value: -1\n",
       "  }\n",
       "  requestPath {\n",
       "    key: \"classifier\"\n",
       "    value: \"seldonio/mock_classifier:1.0\"\n",
       "  }\n",
       "  requestPath {\n",
       "    key: \"outlier-detector\"\n",
       "    value: \"seldonio/outlier_mahalanobis:0.3\"\n",
       "  }\n",
       "}\n",
       "data {\n",
       "  names: \"proba\"\n",
       "  tensor {\n",
       "    shape: 1\n",
       "    shape: 1\n",
       "    values: 0.09630682149894522\n",
       "  }\n",
       "}"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "grpc_request_ambassador(\"mymodel\",API_AMBASSADOR)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "release \"mymodel\" deleted\r\n"
     ]
    }
   ],
   "source": [
    "!helm delete mymodel --purge"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}
